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(57) Abstract 

Disclosed is a computer implemented method for 
automating the selection of an analytical statistical quality 
control (QC) procedure. Inputs of a quality requirement 
and observed performance characteristics of a method of 
measurement are initially provided, together with a data- 
base of theoretical operating characteristics of statistical 
QC procedures. An appropriate quality-planning algo- 
rithm (either analytical or clinical) which relates the qual- 
ity requirement, the observed performance characteristics 
of the method of measurement, and the theoretical oper- 
ating characteristics of statistical QC procedures is also 
provided. Using predetermined QC selection criteria and 
selection logic, a computer is programmed to automati- 
cally select, based on the quality requirement, observed 
method performance, and operating characteristic of can- 
didate QC procedures, a subset of control rules and num- 
bers of control measurements which satisfy selection cri- 
teria and selection logic. 
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AUTOMATIC SELECTION OF STATISTICAL 
QUALITY CONTROL PROCEDURES 

Background of the Invention 

Health care providers in the United States have 
5 entered a new competitive era. Changes in government 
reimbursement policies have limited the amount of money 
that is paid for medical care. Payment of flat fees, 
based on Diagnostic Related Groups (DRGs) , has 
effectively limited hospital and laboratory charges. 

10 Support of Health Maintenance Organizations and similar 
prepaid medical plans has created new health-care 
organizations to compete with traditional providers. The 
new competitive marketplace puts pressure on a provider 
not only to reduce the costs of products and services but 

15 also, and at the same time, to improve their quality, to 
maintain or increase their share of the market. The 
effect on clinical laboratories is that performing more 
tests no longer guarantees more revenue or increased 
resources to support new tests and services. In 

20 addition, the quality of the laboratory's testing 

services that employ complex analytical processes, and 
their usefulness to the physician and patient must be at 
least maintained and, in many cases, improved. 

In general, an analytical process includes a 

25 measurement procedure and a control procedure. The 

measurement procedure refers to the analytical method per 
se, i.e., the reagents, the instrument, and the step-by- 
step directions for producing an analytical result. The 
control procedure refers to that part of the process 

30 concerned with testing the validity of the analytical 
result to determine whether it is reliable and can be 
reported. The development and theoretical aspects of a 
control procedure are discussed, for example, by Westgard 
and Barry (Westgard, J.O. and Barry, P.L. , Cost-Effective 

35 Quality Control: Manage the quality and productivity of 
analytical processes, AACC Press, Washington, DC (1986)). 
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Improvements in an analytical process translate into 
reduced costs, higher productivity and improved quality 
of analysis for an implementing laboratory. In today's 
competitive environment, laboratories which focus on 
5 improvements in such analytical processes are at a 

competitive advantage relative to those which do not. 

Summary of the Invention 

The subject invention relates to a computer 
implemented method for automating the selection of an 

10 analytical statistical quality control (QC) procedure. 

Inputs of a quality requirement and observed performance 
characteristics of a method of measurement are initially 
provided, together with a database of theoretical 
operating characteristics of statistical QC procedures, 

15 An appropriate quality-planning algorithm (either 
analytical or clinical) which relates the quality 
requirement, the observed performance characteristics of 
the method of measurement, and the theoretical operating 
characteristics of statistical QC procedures is also 

20 provided. Using predetermined QC selection criteria and 
selection logic, a computer is programmed to 
automatically choosing, based on the quality requirement, 
observed method performance, and operating characteristic 
of candidate QC procedures, a subset of control rules and 

25 numbers of control measurements which satisfy selection 
criteria and selection logic. 

In another embodiment, the subject invention relates 
to a dynamic QC process for automatically changing the 
statistical QC procedure when there are changes in 

30 performance of a method of measurement. In this 

embodiment, real-time statistical estimates of the 
performance characteristics of a method are provided 
through on-line sampling with an analytical instrument 
and this data is used to automatically select a QC 

35 procedure. 
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Brief Description of the Drawings 

Figure 1 represents a logic diagram for automatic 
quality control procedure selection in both clinical and 
analytical formats . 
5 Figure 2 represents a logic diagram for the dynamic, 

real-time application of automatic quality control 
procedure selection. 

Figure 3 represents a quality control procedure 
selection criteria window. 
10 Figure 4 represents a quality control procedure 

selection logic window. 

Description of Preferred Embodiments 

The present invention relates to a computer 
implemented method for automating the selection of a 

15 statistical quality control procedure. The discussion 

which follows relates primarily to the application of the 
invention in the field of clinical diagnosis for human or 
animal healthcare. However, the invention has 
applications in other fields of use where comparatively 

20 small numbers of controls plotted on a statistical 

control chart (e.g., the Levey-Jennings chart), are used 
to determine the acceptability of much larger numbers of 
samples tested by the same analytical method within the 
same time period. 

25 The selection of a statistical quality control 

procedure is an essential element in the development of 
an analytical process. A statistical quality control 
procedure is necessary to alert an analyst when medically 
important errors occur. The control procedure should be 

30 selected on the basis of the needed capabilities for 
error detection, as well as practical considerations 
related to its rate of false alarms, ease of use, 
training requirements, etc. The performance of the 
control procedure should be documented in quantitative 

35 terms, so the chance of detecting the medically important 
errors is known. 
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A direct way to check the quality of an analytical 
sample is by comparison to control samples having 
predetermined characteristics (e.g., concentration of an 
analyte) which behave similar to patient test samples. 
5 Control sample, as used herein, refers to a control 

material or solution which can be purchased, or produced 
by dissolving appropriate concentrations of a solute in 
an appropriate solvent (typically buffered saline) . Such 
materials are widely available as controls for clinical 

10 chemistry analytes. 

The analytical results obtained for control 
solutions (analyzed for purposes of quality control) are 
called control measurements or control observations. 
Such measurements are made using an analytical 

15 instrument. In some control procedures, the individual 
results are plotted and interpreted directly. Many 
procedures, however, require calculations to be performed 
on the control measurements before the data can be used 
to test control status. These derived or calculated 

20 values are called control statistics; for example, the 

mean of a group of six control measurements could be used 
as a control statistic. 

Statistical techniques can be used to determine 
whether the control measurements are different from the 

2 5 known values. Such procedures are known as statistical 
quality-control procedures, or simply, quality-control 
procedures. Control procedures are commonly implemented 
by plotting control measurements (or statistics 
calculated from those measurements) on control charts; a 

30 control chart, therefore, is a statistical control model 
that provides a visual representation to aid 
interpretation. The American Society for Quality Control 
(ASQC) states that a control chart is n a graphical method 
for evaluating whether a process is, or is not, in a 

35 'state of statistical control'". 

By using control charts, analysts make decisions 
about the acceptability of the analytical results from 
individual analytical runs. Aliquots or samples from one 



04/23/2004, EAST version: 1.4.1 



WO 97/42588 



PCT/US97/07517 



or more stable control solutions are analyzed by the 
measurement procedure during each analytical run. The 
results are plotted on a control chart and compared with 
the control limits drawn on the chart; thus the analyst 
5 determines whether an analytical run is in control or 
out-of-control and whether the results for the patients' 
samples in that run are to be reported. 

According to the ASQC, control limits are the 
"limits on a control chart which are used as criteria for 

10 signaling the need for action, or for judging whether a 
set of data does, or does not, indicate a 'state of 
control'". Control limits consist of upper and lower 
limits, which define a range of acceptable values. In 
general, an analytical run is judged to be in control 

15 when the control measurements fall within the control 
limits, and out of control when the control limits are 
exceeded . 

An analytical run can be defined as a period of time 
or series of measurements, within which the accuracy and 

20 precision of the measuring system is expected to be 

stable; between analytical runs, events may occur causing 
the measurement process to be susceptible to variations 
which are important to detect (see also NCCLS Document 
C24-A, Internal Quality Control Testing: Principles and 

25 Definitions, Vol 11 (No. 6), National Committee for 
Clinical Laboratory Standards, 940 West Valley Road, 
Suite 1400, Wayne, PA 19087-1898). The length of an 
analytical run must be defined appropriately for the 
specific analytical system and specific laboratory 

30 application. The manufacturer should recommend run 
length for the analytical system and the user should 
define run length for the specific application. Thus, 
the length of an analytical run depends on the stability 
of the measurement procedure and its susceptibility to 

35 errors. 

However defined, an analytical run will include a 
certain number of patients' and control samples. An 
analytical run is that group of samples for which a 
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decision is to be made concerning the validity of the 
measurements. The number of control measurements per 
run, N, is the actual number of control measurements 
available when judging the control status of the 
5 measurement procedure, N is critical for determining the 
performance characteristics of a control procedure* 



i. Analytical Quality control 

QC reguirements can be defined in either clinical or 
analytical formats (or terms) and the methods and 

10 compositions of the present invention relate to both 
categories. The logic for both the analytical and 
clinical embodiments are shown in Fig. 1. Referring 
first to the analytical embodiment, inputs of a quality 
requirement are essential. Analytical quality 

15 requirements have been defined by the CLIA-88 proficiency 
testing (PT) criteria for acceptable performance (Federal 
Register 57(40): 7002 (February 28, 1992)). These 
criteria are presented in three different ways: 



• as absolute concentration limits, e.g., target 
20 value ± lmg/dL for calcium; 

• as a percentage, e.g., target value ± 10% for 
albumin, cholesterol, and total protein; and 

• as the distribution of a survey group, e.g., 
target value ± 3 standard deviations (SD) for 

2 5 thyroid stimulating hormone. 



In a few cases, two sets of limits are given, e.g., the 
glucose requirement is given as the target value ± 6 
mg/dL or ± 10% (whichever is greater) . 

The CLIA PT criteria specify the total errors that 

30 are allowable. The total error format is implicit 

because the CLIA-88 rules specify that only a single test 
is to be performed for each PT specimen. Under such 
conditions, the observed analytical error will be the 
total error due to both inaccuracy and imprecision. 

35 Following is a list of the CLIA proficiency testing 
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criteria for the currently regulated analytes, based on 
the February 28, 1992, Federal Register, 
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Routine 

Test or Analyte 

Alanine aminotransferase (ALT) 
Albumin 
5 Alkaline phosphatase 
Amylase 

Aspartate aminotransferase 
(AST) 

Bilirubin, total 

10 Blood gas p0 2 
Blood gas pC0 2 

Blood gas pH 
Calcium, total 
Chloride 
15 Cholesterol, total 
Cholesterol (HDL) 
Creatine kinase 
creatine kinase isoenzymes 

Creatinine 

20 Glucose 

Iron, total 

Lactate dehydrogenase (LDH) 
LDH isoenzymes 

Magnesium 
25 Potassium 
Sodium 

Total protein 
Triglycerides 
Urea Nitrogen 

30 Uric acid 



Chemistry 

Acceptable Performance 

Target value ± 20% 
Target value ± 10% 
Target value ± 30% 
Target value ± 30% 
Target value ± 20% 
Target value ± 0.4 mg/dL or 

± 20% (greater) 
Target value ± 3 SD 
Target value ± 5 mm Hg or 

* 8% (greater) 
Target value ± 0,04 
Target value ± 1.0 mg/dL 
Target value ±5% 
Target value t 10% 
Target value ± 30% 
Target value ± 30% 
MB elevated (present or absent) 
or 

Target value i 3 SD 
Target value ± 0.3 mg/dL 

or ± 15% (greater) 
Target value ± 6 mg/dL 

or ± 10% (greater) 
Target value ± 20% 
Target value t 20% 
LDH 1/ LDH 2 (+ or -) 

or Target value ± 30% 
Target value ± 25% 
Target value ±0.5 mmol/L 
Target value ± 4 mmol/L 
Target value ± 10% 
Target value ± 25% 
Target value ± 2 mg/dL 

or ± 9% (greater) 
Target value ± 17% 



Toxicology 



Test or Analyte 

Alcohol, blood 
Blood lead 

35 Carbamazepine 
Digoxin 

Ethosuximide 

Gentamicin 

Lithium 

40 Phenobarbital 
Phenytoin 
Primidone 

Procainamide (and metabolite) 
Quinidine 
45 Theophylline 
Tobramycin 
Valproic acid 



Test or Analyte 

50 Cell identification 



Acceptable Performance 

Target value ± 25% 
Target value 

or ± 4 
Target value _ 
Target value ± 20% 

or 0.2 
Target value 
Target value 



10% 

mcg/dL (greater) 
± 25 



ng/mL (greater) 
± 20% 
± 25% 



Target value ± 0.3 mmol/L 

or ±20% (greater) 
Target value ± 20% 
Target value 
Target value 
Target value 
Target value 
Target value 
Target value 
Target value ± 25% 

itology 

White cell differentiation 

Erythrocyte count 



25% 
25% 
25% 
25% 
25% 
25% 
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Hematocrit 
Hemoglobin 
Leukocyte count 
Platelet count 
Fibrinogen 

Partial thromboplastin time 
Prothrombin time 



Acceptable Performance 

90% or greater consensus on 

identification 
Target ± 3 SD based on 
percentage of different 
types of white cells 
Target t 6% 
Target t 6% 
Target ±7% 
Target ± 15% 
Target ± 25% 
Target ± 20% 
Target ± 15% 
Target t 15% 



Endocrinology 



Test or Analyte 
10 Cortisol 

Free thyroxine 

Human chorionic gonadotropin 

T 3 uptake 
Triiodothyronine 
15 Thyroid stimulating hormone 
Thyroxine 



Acceptable Performance 
Target value ± 25% 
Target value 1 3 SD 
Target value i 3 SD 

or (positive or negative) 
Target value ± 3 SD by method 
Target value ± 3 SD 
Target value ± 3 SD 
Target value ± 20% 

or 1.0 mcg/dL (greater) 



General Immunology 



Test or Analyte 
Alpha-1 antitrypsin 
2 0 Alpha-f etoprotein 

Antinuclear antibody 

Antistreptolysin o 

Anti-Human Immunodeficiency 
virus 

2 5 Complement C3 

Complement C4 

Hepatitis (HBsAg, anti-HBc, 
HBeAg) 

igA 

3 0 IgB 

IgG 
IgM 

Infectious mononucleosis 
Rheumatoid factor 
35 Rubella 



Acceptable Performance 
Target value ± 3 SD 
Target value i 3 sd 
Target value ± 2 dilution 

or (positive or negative) 
Target value ± 2 dilution 

or (positive or negative) 
Reaction or nonreactive 
Target value i 3 SD 
Target value i 3 SD 
Reactive (positive) or 

nonreactive ( negative ) 
Target value 1 3 SD 
i 3 SD 
± 25% 
i 3 SD 

± 2 dilution 
or (positive or negative) 
Target value t 2 dilution 

or (positive or negative) 
Target value ± 2 dilution 

or (positive or negative) 



Target value 
Target value 
Target value 
Target value 



04/23/2004, EAST Version: 1.4.1 



WO 97/42588 



PCMJS97/07517 



-10- 
In addition to the input of a quality requirement 
(as, for example , defined by the current CLIA standards) , 
inputs of observed performance characteristics of a 
method of measurement are also provided in connection 
5 with the analytical embodiment of the present invention. 
In preferred embodiments, these observed performance 
characteristics include the following: stable imprecision 
observed (s mea5 ) ; stable inaccuracy (bias^) ; matrix 
inaccuracy (bias wtx ) ; number of replicate samples (n &aflp ) ; 
10 z-value for maximum defect rate (z) ; and expected 

instability (frequency of errors, f ) . These terms are 
defined as follows: 



• b i as meas ^ s the stable measurement inaccuracy; 

• bias^^ is the inaccuracy due to matrix effects; 
15 • is the stable measurement imprecision; 

• n sa|ip is the number of replicate samples analyzed 
and averaged to provide a single test result; 

• z is the z value, usually set as 1,65 to 
specify that an analytical run will be rejected 

2 0 when the defect rate is a maximum of 5%; and 

• f is the expected instability (or frequency of 
errors) which is used to indicate how often a 
given method has problems. 



Using these input values, the computer is programmed 
25 to solve the analytical quality planning algorithm set 
forth below: 



TE PT ==biaS mea S +biaS ,natx + ( ASE cont S ro eas) ( ZS **a Accent) /Vn^ 

where TE pT is the analytical quality requirement referred 
to above in connection with the CLIA standards. The 
30 analytical quality requirement is in the form of an 

allowable total error, which is generally represented by 
the abbreviation TE e . In the United States, this term can 
be replaced with TE PT to represent the CLIA proficiency 
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testing criteria for acceptable performance, which are 
analytical quality requirements of the total error form. 
Other terms in the analytical planning algorithm, not 
previously defined, include: 

5 • ASE cont is the change in systematic error (unstable 

inaccuracy) detectable by the QC procedure; 

and 

• ARE cont is the change in random error (unstable 
imprecision) detectable by the QC 
10 procedure . 

In preferred embodiments of the present invention, 
default values and simplifying assumptions are 
established to facilitate application of the algorithm. 
For general application, particularly with automated or 

15 mechanized analyzers, the quality planning algorithm may 
be simplified by combining the two bias terms to specify 
only a total bias, Bias totl , and b) setting ARE com to 1.0 
to optimize for systematic error rather than random 
error. The size of the systematic error that must be 

20 detected to maintain quality within the specified 

requirement can be determined by solving for ASE . then 

cont ' 

letting ASE cont « ASE crit to denote this critically sized 
error . 

RE optimization can be solved optionally. In the 
25 preferred embodiment of the software, this requires 

activation through File and Preferences, ASE cont is set to 
0.0 to simplify the model and only the effect of unstable 
RE is considered. 

Thus, the simplifications of the analytical quality 
30 planning algorithm discussed above are used to calculate 
the critical sizes of systematic and random errors that 
need to be detected so the quality requirement will not 
be exceeded. Under these conditions, the term ASE cHt 
replaces ASE cont and ARE cMt replaces ARE cont . 
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The critical systematic error, ASE crit , is calculated 
as follows: 

ASE crU =[TE PT - | bias^+bias^ | ) ( Vn sanp / S|neas ) ]-z 

The critical random error, ARE crit , is calculated as 
5 follows: 

ARE cHt =TE PT - | bia SmMS +bias Mtx | ) (Vn sanp /z S|neas ) 

Given an appropriate control rule, and the 
information calculated above from the analytical quality 
planning algorithm or algorithms derived from same, data 

10 can be displayed graphically in a manner which 

facilitates optimization of a quality control procedure. 

A control rule is a decision criterion for 
interpreting control data and making a judgment on 
control status (in control or out-of -control) (see e.g., 

15 Westgard et al. , Clin. Chem. 23: 1857 (1977)). The 

theoretical operating characteristics of control rules 
and numbers of control measurements are described by 
power curves or probabilities for rejecting runs having 
different amounts of error present (see e.g., Westgard, 

20 J.O., and Groth, T, Clin. Chem. 25: 394 (1979)). These 
power curves can be calculated based on probability 
theory or, alternatively, determined by computer 
simulation studies (see e.g., Westgard and Groth, Clin. 
Chem. 27: 1536 (1981)) . 

25 For example, the following control rules may be 

employed with individual-value charts where statistical 
control limits are drawn at certain multiples of the 
standard deviation (s) and measurements are plotted 
directly on the chart to assess control status: 

30 * indicates a run is rejected if 1 control value in 

the group of N control measurements exceeds control 
limits set as the mean + or - 3s, where the mean and 
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standard deviation are determined for the control 
material being tested. 

• 2 2s indicates a run is rejected if 2 consecutive 
control values exceed the same control limit , which 

5 is either the mean + 2s or the mean - 2s. 

• R 4s is a range rule where a run is rejected if one 
control value exceeds the mean + 2s and another 
exceeds the mean - 2s. 

• 4 1s indicates a run rejection if 4 consecutive 

10 control values exceed the mean + is or the mean - 

is, 

• 10 x refers to a criterion where a run is rejected if 
10 consecutive control values fall on one side of 
the mean. 



15 Other control rules establish control limits having 

a specified probability for false rejection (P fr ) . The 
actual control limits are determined by looking up 
factors in a table, then multiplying the standard 
deviation by those factors. Some examples are the 

20 following: 



• 1 o.oi refers to a control chart where individual 
values in a group of N control measurements are 
plotted directly and compared to control limits set 
such that the probability for false rejection is 

25 0.01, i.e., there is only a l% chance that a good 

run will be falsely rejected. 

• x o.oi re fers to a mean chart where the mean of a group 
of N control measurements is plotted and compared to 
control limits set such that the probability for 

30 false rejection is 0.01. 

• ^.oi "fers to a range chart where the difference 
between the high and low observations in a group of 
N control measurements is compared to a control 
limit set such that the probability for false 

35 rejection is 0.01. 
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Combinations of rules are indicated by individual 
rule abbreviations connected by a slash character, for 
example, l 3s /2 2s indicates a multi-rule QC procedure that 
is made up of l 3s and 2 2s control rules, and X 0>01 /R 0-01 
5 indicates a QC procedure using both mean and range 
control charts. 

For most control rules in common use in clinical 
laboratories, the control limits are set as the mean plus 
and minus constant multiples of the observed standard 

10 deviation, regardless of the number of control 

measurements being collected (N) . However, for mean and 
range rules and single-value variable-limit rules, the 
control limits change as N changes. The actual control 
limits are calculated by multiplying the standard 

15 deviation by the factor given in the table below. 



Factors for calculating control limits 



Rule 


Number of control measurements (N) 


•^-o.os 


2.24 


2.39 


2.50 


2.64 


2.74 


1 

^0.01 


2.81 


2.93 


3.01 


3.13 


3.21 


^0.002 


3.27 


3.36 


3.44 


3.52 


3.59 


X 0.05 


1.39 


1.13 


0. 98 


0.80 


0. 60 


X 0.01 


1.82 


1.49 


1.29 


1.05 


0.91 


X 0.002 


2.19 


1.78 


1.54 


1.26 


1.09 | 


^0.05 


2.77 


3.31 


3.63 


4.03 


4.29 


^.01 


3.64 


3.63 


4.40 


4.76 


4.99 


R 0.Q0? 


4.37 


4.80 


5.05 


5.37 


5.58 



In the computer-implemented method of the present 
invention, a database of operating characteristics is 
provided for a wide variety of control rules and classes 

30 of control rules, including single-rule fixed limit, 
single-rule variable limit, multirule, and mean and 
range. In addition, an editor function allows the user 
to incorporate the operating characteristics of 
additional control rules and numbers of control 

35 measurements by entry of the probabilities of rejection 
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for specified increases in systematic (for ASE cont values 
of 0.0, 0„5s, 1.0s, 1.5s, 2.0s, 3.0s, 4.0s/ ARE cont values 
of 1.0, 1.5, 2.0, 3.0, 4.0), as determined from 
probability theory or by computer simulation studies. 
5 These operating characteristics may be represented 

graphically using a variety of visual tools which are 
known in the art. These visual tools include, for 
example, power function graphs, critical error charts and 
operating specification charts (referred to herein as 

10 OPSpecs charts) . Power function and critical error 

graphs plot the probability for rejection (P) on the y- 
axis and ASE (or ARE when optimizing for random error) on 
the x-axis. Critical error graphs display the size of 
the calculated medically important or critical systematic 

15 or random errors on the power curves. OPSpecs charts 
plot allowable inaccuracy (bias totl ) on the y-axis and 
allowable imprecision (sj on the x-axis. For a more 
detailed discussion of OPSpecs charts see, for example, 
Westgard, Arch Pathol. Lab. Med. 116: 765 (1992); 

20 Westgard, Clin. Chem. 38: 1226 (1992); and Westgard, 

J.O., "Operating specifications for precision, accuracy, 
and quality control", OPSpecs Manual — Expanded Edition: 
Westgard Quality Control Corporation, Ogunquit ME, 
(1996) . 

25 Automatic QC (i.e., automatic control rule) 

selection is implemented on the basis of defined 
selection criteria and selection logic. In preferred 
embodiments, selection criteria employed to select 
candidate control rules from the database include 1) type 

30 of control rule; 2) total number of control measurements 
per run; 3) probability of false rejections (also 
expressed in percent as a false rejection rate); and 4) 
probability of error detection (also expressed in percent 
as error detection rate, or as percent analytical quality 

35 assurance (AQA) ) . Those skilled in the art are familiar 
with all of these parameters, and their significance with 
respect to control rule selection. 
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In preferred embodiments, default settings are 
provided. These default settings are shown in the 
selection criteria window of Fig, 3. Users are, of 
course, free to change the default settings to define the 
5 types of control rules to be implemented (e.g., single 
rule fixed limits, single rule variable limits, multi- 
rule, mean/range rules) , the total number of control 
measurements of interest (N's of 1, 2, 3, 4, 6 and/or 8), 
the maximum false rejection (10%, 5% or 1%), and the 

10 desired error detection rate (90%, 50%, 25%) (%AQA) for 
high, moderate and low instability. For example, with 
default settings, the program first attempts to select QC 
procedures having at least 90% error detection, 
regardless of the expected frequency of errors, but will 

15 select QC procedures having at least 50% error detection 
if an expected instability of less than 2% has been 
entered. By changing the default settings, a user can 
enable the program to select QC procedures having at 
least 2 5% error detection for an expected instability of 

20 less than 2%. 

One of skill in the art will recognize that the 
options provided in the selection criteria window of Fig. 
3 are somewhat arbitrary. For example, the desired error 
detection percentages, and minimum false rejection 

25 percentages provided could be percentages other than 
those provided. 

Selection logic refers to the preferences, definable 
by the user, which determine the priority applied by the 
computer software in implementing the selection criteria 

30 discussed above. Default settings are again provided, as 
shown in Fig. 4. Users can modify default settings to 
define preference for low N vs. high N, single rule vs. 
multi-rule, and lower cost vs. higher error detection. 
For example, when 90% error detection can be achieved, 

3 5 default settings set preferences for the lowest N, single 
rule procedures which are the least expensive options. 
However, if 90% error detection can not be achieved, the 
default settings for 50% error detection provide 
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preferences for the highest N multi-rule procedures 
having the highest error detection. As was stated in 
connection with selection criteria, the options provided 
in the selection logic window shown in Fig, 4 are merely 
5 examples- Those skilled in the art could design 

alternatives without the application of inventive skill. 
Given required inputs for QC selection criteria and 
selection logic, the computer searches the control rule 
library for control rules and Ns satisfying selection 
10 criteria. The limits of allowable imprecision and 

inaccuracy for an OPSpecs chart are then derived for the 
candidate control rules and compared to the observed 
bias totl and s (Beas of the method (defined as the operating 
point) . The 90% AQA charts are first checked to identify 
15 control rules and N whose operating limits are above the 
operating point . Each of these rules correspond to an 
appropriate QC procedure. 

The computer is further programmed for control rule 
optimization. Control rule optimization means selecting: 
20 1) an appropriate control rule from the database which 
has a high probability of detecting a systematic or 
random error in the patient samples results; 2) a low 
probability of false rejection (i.e., having to falsely 
retest the entire patient sample run due to a statistical 
25 performance characteristic of the control rule); and 3) a 
minimum number of controls in the run of patient samples. 

If none of the control rules provides a solution at 
90% AQA, the computer is programmed to check 50% AQA 
charts. If control rule solutions are identified, 
30 optimization is carried out as described above. If no 
solutions are identified at 50% AQA, 25% AQA charts are 
checked. If solutions are identified, control rule 
optimization is carried out. If no solutions are 
identified, the analyst is required to make a manual 
35 selection. 

When the computer completes optimization by the 
selection of a control rule and N, the analyst must 
review this selection and initiate documentation, or 
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override this selection by making a manual selection of a 
different rule and N. This option for review and manual 
selection could be eliminated, but is considered 
desirable in order to maintain the professional interest 
5 and responsibility of laboratory scientists. This 

selection completes the automatic QC selection process 
and provides the user with the information to prepare a 
control chart that can be used to plot control 
measurements from routine analytical runs. The control 
10 rule selected dictates whether the patient sample 

measurements are judged to be in a state of control, or 
not. 

Given the subject disclosure , it would be a matter 
of routine skill to select an appropriate computer system 
15 and implement the claimed process on that computer 

system. This statement applies to all embodiments of the 
present invention, including the dynamic, real-time 
application discussed below. 

ii. Clinical Quality control 

2 0 As stated previously, QC requirements can be defined 

in either clinical or analytical formats. Having 
discussed the analytical format, the clinical format will 
now be addressed. The logic for the clinical embodiments 
of the subject computer-implemented methods (as well as 

25 the analytical embodiments) , are shown in Fig. 1. 
Clinical quality can be described as a medically 
important change in a test result that would affect the 
clinical interpretation of the test. The term decision 
interval (D lnt ) is used to represent the size of the 

30 change between two different test results that would 
cause different actions or outcomes in the medical 
application of the test result (Westgard et al., Clin. 
Chem. 37: 656 (1991)). 

The clinical quality-planning model that is 

3 5 incorporated in preferred embodiments of the present 

invention provides an approach which considers the 
potential variation of a test result due to biologic, 
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pre-analytic, analytic, and quality control factors — all 
of which may contribute to the uncertainty of a single 
test result. The use of clinical quality requirements 
can be expected to be difficult because of the difficulty 
5 in finding the necessary information for all the terms 
included in the model. 

The most readily available recommendations are those 
provided by Skendzel et al. (Am. J". Clin. Pathol. 83: 200 
(1985)). It is noted that information concerning 

10 decision intervals is found in Table 1 of Skedzel et al., 
not in the table relating to medically allowable standard 
deviations, whose calculations do no not take into 
account the within-subject biological variation. 
Quality planning algorithms describe the 

15 mathematical relationship between the quality of a test 
result and the factors that can cause variation in that 
result. Some of the factors which roust be considered are 
analytical. These include, for example, the imprecision 
and inaccuracy of the measurement procedure and the 

20 sensitivity (or error detection capability) of the QC 
procedure. Others are pre-analytical, such as the 
within-subject biological variation that describes the 
changes in concentration due to normal biological 
variation. The clinical quality planning algorithm 

25 employed in preferred embodiments of the present 
invention is shown below. 
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D Int = bias epec + bias meas + bias matx + ASE cont S measAl^Ta 



samp 



+Z 



s2 wsub s2 B pec (ARE cont s meas )2 
n test n test n spec n test n spec n samp 



The clinical quality planning algorithm contains 
additional parameters , as compared with the analytical 
quality planning algorithm. Generally, these additional 
5 parameters relate to pre-analytical components of test 
variation. The clinical quality planning algorithm 
variables are defined as follows; 

• bias spec represents sampling or specimen bias which 
allows for a pre-analytical component of systematic 

10 error which is entered as a percentage; 

• bias^^ is the inaccuracy due to matrix effects; 

• s wub represents the within-subject biological 
variation which is the standard deviation that 
represents the average biological variation of an 

15 individual subject which is entered as a percentage; 

• s 8pec is the between specimen sampling variation which 
allows for the random variation between specimens, 
which is entered as a percentage; 

• n test is the number of tests drawn at separate times 
20 which are averaged to obtain a test result that will 

be interpreted; 

• n spec * s the number of specimens which are drawn and 
averaged for each test; 

• n sanp is the number of samples or replicates measured 
25 for each specimen (see page 8); and 

• z-value defines the maximum defect rate that is 
allowed before the testing process is declared out- 
of-control (the default value is set at 1.65 which 
defines a maximum defect rate of 5%) . 
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Equations for the calculation of the critical 
systematic and random errors are derived in the same 
manner as was described for the analytical model. The 
equations will, of course, appear to be considerably more 
5 complex due to the inclusion of the additional terms 

defined above* Of the pre-analytical terms, it is most 
critical to account for within-subject biological 
variation (see e.g., Fraser, C.G. , Arch. Pathol. Lab. 
Med.: 916 (1992)). Estimates of other terms may also be 

10 found in the literature (see e.g., Young, Effects of 
Preanalytical Variables on Clinical Laboratory Tests, 
AACC Press, Washington, DC (1993)). 

Automatic QC selection in the clinical embodiment of 
the present invention is a process which is identical to 

15 that described above in connection with the analytical 

embodiment, but for the differences between the clinical 
and analytical algorithms. 

iii. Dynamic, Real-Time Application of Automatic QC 
Selection 

20 Automatic QC selection in the context of both 

analytical and clinical quality planning has been 
discussed above. Figure 2 shows the logic for a dynamic, 
real-time application of automatic QC selection which can 
be applied in either a clinical or analytical context. 

25 The initial steps in this application are identical to 

those discussed previously in sections i) and ii) , above. 

For example, with respect to an analytical 
application of the dynamic embodiment of the present 
invention, initial inputs of a quality requirement and 

30 observed performance characteristics of a method are 
provided. A quality control procedure (comprising a 
control rule and an N) is then selected automatically as 
previously described. As shown in Fig. 2, quality 
control data, generated by an analytical instrument, are 

35 transferred to a software program, either embedded within 
the medical device micro processor, or to a laboratory 
information system (LIS) , or data management system 
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(DMS) , or a stand alone PC program resident on a PC hard 
disk. The integration of these various components is a 
matter of routine skill. The control data is then 
converted to a control chart format in real time. The 
5 control data is then analyzed on a continuing basis, and 
analytical statistical method of performance results are 
calculated. Numerical values for the method performance 
are input into a computer programmed to carry out the 
automatic QC procedure selection described above. In 
10 this way, an appropriate QC procedure is automatically 

selected. The QC procedure which has been automatically 
selected is used to evaluate the control data for 
purposes of accepting or rejecting the patient samples in 
a run. 

!5 The introduction of a feedback loop for real time QC 

data enables an analytical measurement system, for 
example, to automatically select a higher N value, or a 
more sensitive control rule if method performance 
deteriorates. Alternatively, N would be automatically 

2 0 decreased or a less sensitive control rule would be 
automatically selected if method performance were to 
improve . 
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CLAIMS 

1. A computer implemented method for automating the 
selection of an analytical statistical quality 
control <QC) procedure, the method comprising: 

a) providing inputs of a quality requirement and 
observed performance characteristics of a 
method of measurement; 

b) providing a database of theoretical operating 
characteristics of statistical QC procedures; 

c) providing an analytical quality-planning 
algorithm which relates the quality 
requirement, the observed performance 
characteristics of the method of measurement, 
and the theoretical operating characteristics 
of statistical QC procedures; 

d) providing QC selection criteria and selection 
logic; 

e) automatically choosing, based on the quality 
requirement, observed method performance, and 
operating characteristic of candidate QC 
procedures, a subset of control rules and 
numbers of control measurements which satisfy 
selection criteria and selection logic. 

2. The method of Claim 1 wherein the statistical QC 
procedure can be characterized by a control rule, 
control statistic, or control chart whose operating 
characteristics can be described in terms of 
probabilities of acceptance or rejection as a 
function of the size of systematic and random 
errors. 

3. The method of Claim 2 wherein the control statistic 
represents the average of normal patient test 
results used to monitor errors or changes. 
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4. The method of Claim 1 wherein the quality 
specification is an analytical quality requirement 
for the allowable total error, TEa. 

5. The method of Claim 1 wherein the input parameters 
necessary for the analytical quality-planning 
algorithm are the stable imprecision observed (s^) ; 
stable inaccuracy (bias^) ; matrix inaccuracy 
(bias^); number of replicate samples (n SBmp ) ; and z- 
value for maximum defect rate (z) . 

6. The method of Claim 1 further comprising 
automatically selecting from among the subset of 
control rules in step e) the control rule which best 
satisfies the selection criteria and selection logic 
for type of control rule or control chart, total 
numbers of control measurements per analytical run, 
error detection rate, and false rejection rate. 



7. The method of Claim 6 wherein the database of 
operating characteristics of statistical QC 
procedures comprises types of control rules and 
control charts selected from the group consisting of 
single-rule constant limits, single-rule variable 
limits, multi-rule with specified maximum number of 
runs, and mean/range rules. 



8. The method of Claim 6 wherein the selection criteria 
and selection logic for the total number of control 
measurements per analytical run (N) includes 1, 2, 
3, 4, 6, and 8 control measurements per run. 

9. The method of Claim 6 wherein the selection criteria 
and selection logic for the rate of error detection 
are 90% or more, 50% or more, or 25% or more. 
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10. The method of Claim 9 wherein the selection criteria 
and selection logic for error detection depend on 
the input for the observed instability of the 
measurement procedure. 

11. The method of Claim 6 wherein the selection criteria 
and selection logic for the rate of false rejection 
are 10% or less, 5% or less, or 1% or less, 

12. A computer implemented method for automating the 
selection of a clinical statistical quality control 
(QC) procedure, the method comprising: 

a) providing inputs of a quality requirement and 
observed performance characteristics of a 
method of measurement; 

b) providing a database of theoretical operating 
characteristics of statistical QC procedures; 

c) providing a clinical quality-planning algorithm 
which relates the quality requirement, the 
observed performance characteristics of the 
method of measurement, and the theoretical 
operating characteristics of statistical QC 
procedures; 

d) providing QC selection criteria and selection 
logic; 

e) automatically choosing, based on the quality 
requirement, observed method performance, and 
operating characteristic of candidate QC 
procedures, a subset of control rules and 
numbers of control measurements which satisfy 
selection criteria and selection logic. 

13. The method of Claim 12 wherein the statistical QC 
procedure can be characterized by a control rule, 
control statistic, or control chart whose operating 
characteristics can be described in terms of 
probabilities of acceptance or rejection as a 
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function of the size of systematic and random 
errors. 

14. The method of Claim 13 wherein the control statistic 
represents the average of normal patient test 
results used to monitor errors or changes. 

15. The method of Claim 12 wherein the quality 
specification is a clinical quality requirement for 
a medically important change in the form of a 
decision interval, Dint. 

16. The method of Claim 12 wherein the input parameters 
necessary for the clinical quality-planning 
algorithm are the stable imprecision observed (s^) ; 
stable inaccuracy (bias^) ; matrix inaccuracy 
(bias^) ; specimen bias <bias gpec ) ; within-subject 
biological variation (s ttSUb ) ; sampling variation 
(s spec ) ; number of tests (n test ) ; number of specimens 
(n spec ) ? number of replicate samples (n saiTp ) ; and z- 
value for maximum defect rate (z) . 

17. The method of Claim 12 further comprising 
automatically selecting from among the subset of 
control rules in step e) the control rule which best 
satisfies the selection criteria and selection logic 
for type of control rule or control chart , total 
number of control measurements per analytical run, 
error detection rate, and false rejection rate. 

18. The method of Claim 17 wherein the database of 
operating characteristics of statistical QC 
procedures comprises types of control rules and 
control charts selected from the group consisting of 
single-rule constant limits, single-rule variable 
limits, multi-rule with specified maximum number of 
runs, and mean/range rules. 
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19. The method of Claim 17 wherein the selection 
criteria and selection logic for the total number of 
control measurements per analytical run (N) includes 
1, 2, 3, 4, 6, and 8 control measurements per run. 

20. The method of Claim 19 wherein the selection 
criteria and selection logic for the rate of error 
detection are 90% or more, 50% or more, or 2 5% or 
more. 

21. The method of Claim 20 wherein the selection 
criteria and selection logic for error detection 
depends on the input for the observed instability of 
the measurement procedure. 

22. The method of Claim 16 wherein the selection 
criteria for the rate of false rejection are 10% or 
less, 5% or less, or 1% or less. 

23. A dynamic QC process for automatically changing the 
statistical QC procedure when there are changes in 
performance of the method of measurement, the method 
comprising: 

a) providing inputs of a quality requirement and 
initial observed performance characteristics of 
a method of measurement; 

b) providing a database of theoretical operating 
characteristics of statistical QC procedures; 

c) providing quality-planning algorithms which 
relate the quality requirement, the observed 
performance characteristics of the method of 
measurement, and the theoretical operating 
characteristics of statistical QC procedures; 

d) providing QC selection criteria and selection 
logic; 

e) automatically choosing, based on the quality 
requirement, observed method performance, and 
operating characteristic of candidate QC 
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procedures, a subset of control rules and 
numbers of control measurements which satisfy 
selection criteria and selection logic [for the 
desired rate of error detection and the desired 
rate of false rejection]. 

f) providing real time statistical estimates of 
method performance characteristics through on- 
line sampling with an analytical instrument; 

g) repeating steps b)-f) above for a sequence of 
runs thereby automatically selecting a higher 
number of control measurements or more 
sensitive control rules when method performance 
deteriorates and decreasing the number of 
control measurements or changing to less 
sensitive control rules when method performance 
improves ; 

h) implementing the selected statistical QC 
procedure through calculations and control 
charts; 

i) providing an assessment of the state of control 
and a decision about the acceptance or 
rejection of individual analytical runs based 
on the selected statistical QC procedure. 
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